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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 
• Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)|3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) |3 Claim(s) 1-48 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) E3 Claim(s) 12-20.33 and 36-38 is/are allowed. 

6) 12 Claim(s) 1.2.4-11.21.23.24.26-32.34.35.39-41 and 46-48 is/are rejected. 

7) I3 Claim(s) 3.22.25 and 42-45 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) 12 The drawing(s) filed on 13 July 2000 is/are: a)D accepted or b)[2 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§ 119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)DAII b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 
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Drawings 

The drawings are objected to under 37 CFR 1.83(a). The drawings must show every 
feature of the invention specified in the claims. Therefore, the limitations of claims 4, 5, 18, and 
19 must be shown or the feature(s) canceled from the claim(s). No new matter should be 
entered. 

A proposed drawing correction or corrected drawings are required in reply to the Office 
action to avoid abandonment of the application. The objection to the drawings will not be held 
in abeyance. 

Allowable Subject Matter 

Claims 12-20, 33, and 36-38 are deemed allowable over the prior art of record, the 
reasons for allowance are as follows: 

In regards to claim 12 the prior art of record does not disclose nor preclude an image 
capture system providing an extended effective dynamic range comprising, an image sensing 
device having standard photosites with a predetermined standard response to a light exposure 
and non-standard photosites with a slower response to the same light exposure, an optical 
section, mans for converting the image signal into digital image signals, and a processor that 
processes the digital image signals against a plurality of thresholds, including a high exposure 
response threshold for the standard photosites and a low exposure response threshold for the non- 
standard photosites, replaces the digital image signals from standard photosites exceeding the 
high exposure response threshold with a combination of the digital image signals from a 
neighborhood of non-standard photosites, and replaces the digital image signals from non- 
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standard photosites less than the low exposure response threshold with a combination of the 
digital image signals from a neighborhood of standard photosites. Examiner notes that the 
bellow referenced Park patent does not disclose replacing either standard or non-standard pixels 
from a neighborhood of the opposite, merely replacing from the adjoining standard or non- 
standard pixel. The bellow referenced Mayar article discloses replacing standard photosites 
using a neighborhood of non-standard photosites but does not disclose replacing non-standard 
photosites using a neighborhood of standard photosites. Examiner notes that this is because of 
the standard and non-standard pixel pattern disclosed by Mayar in Fig. 1 . Examiner notes that 
Mayar discloses that the pattern can take other forms however does not disclose other possible 
forms such as Applicant's patterns disclosed in Figs. 3A and 3B. 
Claims 13-20 depend from claim 12. 

In regards to claim 33 the prior art of record does not disclose nor fairly suggest an 
algorithm utilizing the image sensor as claimed in claim 3 1 in order to expand the response of 
the standard photosites of each color to increased exposures by utilizing the image signals from 
neighboring non-standard photosites of the same color at least two lines removed from the 
corresponding standard photosite and to expand the response of the non-standard photosites of 
each color to decreased exposures by utilizing the image signals from neighboring standard 
photosites of the same color at least two lines removed from the corresponding non-standard 
photosite. Examiner notes that while it would be obvious to one of ordinary skill in the art to 
utilize a color filter for the image sensor there is no teaching or suggestion in the prior art of 
record to perform the dynamic range expansion in this way. 



Application/Control Number: 09/6 1 5,398 Page 4 

Art Unit: 2615 

In regards to claim 36 Examiner refers to the reasons for allowance of claim 12 being 
substantially identical 

Claims 37 and 38 depend from claim 36. 

Claims 3, 22, 25, and 42-45 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

In regards to claims 3, 22, 25 Examiner refers to the reasons for allowance for claim 12 
being substantially identical. 

In regards to claim 42 the prior art of record does not disclose nor fairly suggest the 
image capture system as claimed in claim 41, wherein the photosites are color photosites and are 
arranged such that the eight photosites constituting the nearest neighbors of a given standard 
photosite which is green in color comprise four non-standard photosites which are green in color, 
one standard photosite which is red in color, one standard photosite which is blue in color, one 
non-standard photosite which is red in color, and one non-standard photosite which is blue in 
color. Examiner again notes that while it would have been obvious to one of ordinary skill in the 
art to have utilized a color filter for an image sensor there is not teaching or suggestion in the 
prior art of record to provide the pattern depicted in Fig. 3B and claimed here. 

In regards to claims 43-45 similar reasons for allowance can be used, namely that the 
prior art of record does not disclose the pattern depicted in Fig. 3B. 
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Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this 
or a foreign country, before the invention thereof by the applicant for a patent. 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

Claims 1, 2, 6, 8, 10, 1 1, 21, 23, 24, and 46-48 are rejected under 35 U.S.C. 102(b) as 
being anticipated by (USPN 5,714,753 to Park). 

In regards to claim 1 Park discloses an image capture system for generating an extended 
effective dynamic range from a signal provided by an image sensor, said image capture system 
comprising: 

an image sensing device having standard photosites (e.g., element 22 of Fig. 3) with a 
predetermined response to a light exposure and a non-standard photosites (e.g., element 24 of 
Fig. 3) with a slower response to the same light exposure (e.g., the photodiode 24 is smaller, 
therefore it has a slower response to the same light exposure); 

an optical section for exposing the image sensing device to image light, thereby causing 
the image sensing device to generate an image signal (e.g., the claimed optical section is inherent 
with a camera); and 

a processing section for expanding the response of the standard photosites to increased 
light exposures by utilizing the image signals from neighboring non-standard photosites (e.g., the 
processing section is the pixel and its peripheral control logic, wherein as shown in Fig. 5 an 
expanded response of the standard photosites to increased light exposures is created by utilizing 
the image signals from neighboring non-standard photosites; column 4, lines 1-21). 
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In regards to claim 2 Park discloses the image capture system as claimed in claim 1 
wherein the processing section expands the response of the non-standard photosites to decreased 
light exposures by utilizing the image signals from neighboring standard photosites (e.g., when 
the response of the photodiodes 22 is on the curve f of Fig. 5 then the response of the non- 
standard photosites is expanded by the response of the standard photosites, namely the response 
of the standard photosites is used for the pixel value). 

In regards to claim 6, Examiner notes that it is implicit in the Park reference that the 
image sensor is in a digital camera, wherein, as stated above, the processing section is in the 
image sensor. 

In regards to claim 8 Examiner notes that Park does not disclose any color filters, 
therefore the image sensor would be monochromatic. 

In regards to claims 10 and 1 1 see Examiners notes on the rejection of claims 1 and 2. 

In regards to claim 21 see Examiners notes on the rejection of claims 1 and 2. 

In regards to claims 23 and 24 see Examiners notes on the rejection of claims 1 and 2. 

In regards to claim 46 note that the nearest photosite with the same color does not have 
the same response as the given photosite as shown in Fig. 3. For example, the top pixel in Fig. 3 
the nearest photosite to photodiode 22 is photodiode 24. 

In regards to claims 47 and 48 see Fig. 3. Note that Park's invention is a CCD wherein 
the image sensor depicted in Fig. 3 would be expanded two-dimensionally. 



Claims 1, 2, 8, 10, 1 1, 21, 23, 24, 26-30, 34, 46, and 48 are rejected under 35 
U.S.C. 102(a) as being anticipated by ("High Dynamic Range Imaging: Spatially Varying Pixel 
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Exposures" by Shree K. Mayar and Tomoo Mitsunaga. Proceedings IEEE Conference on 
Computer Vision and Pattern Recognition, Vol. I, pp. 472-479), herein Mayar. 

In regards to claim 1 Mayar discloses an image capture system for generating an 
extended effective dynamic range from a signal provided. by an image sensor, said image capture 
system comprising: 

an image sensing device having standard photosites (e.g., photosite e3 depicted in Fig. 1, 
note the first paragraph of section 3 on page 473) with a predetermined response to a light 
exposure and a non-standard photosites (e.g., elements e0-e2 of Fig. 1) with a slower response to 
the same light exposure; 

an optical section for exposing the image sensing device to image light, thereby causing 
the image sensing device to generate an image signal (e.g., the claimed optical section is inherent 
with a camera); and 

a processing section for expanding the response of the standard photosites to increased 
light exposures by utilizing the image signals from neighboring non-standard photosites (e.g., see 
section 6.2). 

In regards to claim 2 Mayar discloses the image capture system as claimed in claim 1 
wherein the processing section expands the response of the non-standard photosites to decreased 
light exposures by utilizing the image signals from neighboring standard photosites (e.g., see Fig. 
6, paragraph 4 of section 7 on page 476). 

In regards to claim 8 not that Maya does not disclose any color filters, therefore the 
photosites are monochromatic. 

In regards to claims 10 and 1 1 see Examiners notes on the rejections of claims 1 and 2. 
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In regards to claim 21 see Examiners notes on the rejections of claims 1 and 2. 

In regards to claims 23 and 24 see Examiners notes on the rejections of claims 1 and 2. 

In regards to claim 26 Mayar discloses an image sensor for generating an image signal 
with a differential response to image light, said image sensor comprising: 

an array of photosites divided into standard photosites and non-standard photosites (e.g., 
Fig. 1); and 

a structural element overlying the photosites and providing the standard photosites with a 
predetermined standard response to a light exposure and the non-standard photosites with a 
slower response to the same light exposure (e.g., paragraph 3 of section 3 on page 473). 

In regard to claim 27 Examiner notes that Mayar discloses "the sensitivity of the pixels 
can be preset by using different microlenses on the array (page 473)". 

In regards to claim 28 Examiner notes that Mayar discloses "the sensitivity of the pixels 
can be preset by . . . embedding different apertures for the potential wells of the pixels (page 
473)". 

In regards to claim 29 Examiner notes that Mayar discloses "One approach is to place a 
mask with cells of different optical transparencies adjacent to the detector array (page 473)". 

In regards to claim 30 see Examiners notes on the rejections above. 

In regards to claim 34 Examiner notes that is inherent to place the CCD described by 
Mayar in a digital camera. 

In regards to claims 46 and 48 see Fig. 1. 
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Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

Claims 7, 9, 39-41 are rejected under 35 U.S.C. 103(a) as being unpatentable over (USPN 
5,714,753 to Park). 

In regards to claim 7 Park does not explicitly disclose that the non-standard photosites 
have a response that is slower by at least one stop compared to the standard photosites. 
Examiner notes that it would have been obvious to one of ordinary skill in the art at the time of 
the invention to have defined the differences in photosite response in terms of a number of 
"stops" since such a definition is well known in the photography art to describe exposure values. 
Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to provide a response of the non-standard (slower response) photosites of Park to be 
slower by at least one stop compared to the standard photosites in order to generate a difference 
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in exposure response between the standard and non-standard photosites and thus extend the 
dynamic range of the pixel as taught by Park. 

In regards to claim 9 Examiner notes that it is extremely well known in the art to provide 
a color filter so as to produce a color image. Official notice is taken. Therefore it would have 
been obvious to one of ordinary skill in the art at the time of the invention to have added a color 
filter to Park's invention in order to generate a color image. Examiner notes that Park discloses 
each pixel has two photosites, wherein by adding the color filter the standard and non-standard 
photosites would be of the same color. 

In regards to claim 39 see Examiners notes on the rejection of claim 9. Note that the 
nearest photosite with the same color does not have the same response as the given photosite. 
For example, in applying a color filter to the top pixel in Fig. 3 the nearest photosite with the 
same color to photodiode 22 is photodiode 24, which has the same color as discussed above. 

In regards to claim 40 see Examiners notes on the rejection of claim 9. Park discloses the 
image capture system as claimed in claim 1, wherein the photosites are arranged such that the 
four photosites constituting the nearest neighbors of a given non-standard photosite comprise two 
standard photosites and two nonstandard photosites (e.g., see Fig. 3. Note that Parks invention is 
a CCD image sensor as disclosed in the background on column 1, lines 8-10, wherein thus be 
extended two-dimensionally) . 

In regards to claim 41 see Examiners notes on claim 40. 

Claims 4-7, 31, and 32 are rejected under 35 U.S.C. 103(a) as being obvious over ("High 
Dynamic Range Imaging: Spatially Varying Pixel Exposures" by Shree K. Mayar and Tomoo 
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Mitsunaga. Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Vol I, 
pp. 472-479). 

In regards to claim 4 Mayar does not explicitly disclose where the processing takes place. 
Examine notes that it is extremely well known to perform image processing functions on an 
external computer so as to reduce the number of parts needed on the digital camera and have 
access to higher processing power. Official notice is taken. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of the invention to have had the processing 
section in a host computer separate from the digital camera in order to reduce the number of 
parts needed on the digital camera and have access to higher processing power. 

In regards to claim 5 see Examiners notes on the rejection of claim 4. Note that it is 
further well known in the art to provide remote access to computers and/or servers from a digital 
camera in order to reduce the number of parts needed on the digital camera and have access to 
higher processing power, while at the same time not be constrained to being in close proximity to 
the computer/server. Official notice is taken. Therefore it would have been obvious to one of 
ordinary skill in the art at the time of the invention to have enabled the processor to be accessed 
via a network in order to enable remote control of processor without being constrained to being 
in close proximity to the computer/server. 

In regards to claim 6 Mayar does not explicitly disclose where the processing takes place. 
Examine notes that it is extremely well known to have all of the processing for a camera to be 
done in the camera so as to enable a camera to be portable and independent from any external 
computer. Official notice is taken. Therefore it would have been obvious to one of ordinary 
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skill in the art at the time of the invention to have had the processing section included in the 
digital camera. 

In regards to claim 7 Mayar does not explicitly disclose that the non-standard photosites 
have a response that is slower by at least one stop compared to the standard photosites. 
Examiner notes that it would have been obvious to one of ordinary skill in the art at the time of 
the invention to have defined the differences in photosite response in terms of a number of 
"stops" since such a definition is well known in the photography art to describe exposure values. 
Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to provide a response of the non-standard (slower response) photosites of Park to be 
slower by at least one stop compared to the standard photosites in order to generate a difference 
in exposure response between the standard and non-standard photosites and thus extend the 
dynamic range of the pixel as taught by Park. 

In regards to claim 3 1 Examiner notes that it is extremely well known in the art to 
provide a color filter so as to produce a color image. Official notice is taken. Therefore it would 
have been obvious to one of ordinary skill in the art at the time of the invention to have added a 
color filter to Park's invention in order to generate a color image. 

In regards to claim 32 Examiner notes that it is extremely well known to use a Bayer 
color filter so as to generate more data for the green color plane. Official notice is taken. 
Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to have used a Bayer color filter in order to generate more data for the green color 
plane. 
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Claim 35 is rejected under 35 U.S.C. 103(a) as being unpatentable over ("High Dynamic 
Range Imaging: Spatially Varying Pixel Exposures" by Shree K. Mayar and Tomoo Mitsunaga. 
Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Vol. I, pp. 472-479) 
in view of (USPN 5,714,753 to Park). 

In regards to claim 35 Examiner notes that Mayar discloses using different micro lenses 
on the array but does not explicitly disclose that standard photosites would have microlenses and 
non-standard photosites would not. Park discloses the use of a microlens to focus light onto 
photodiode 22 in order to increase the response of photodiode 22 as compared to photodiode 24 
(column 2, lines 54-57). Therefore it would have been obvious to one of ordinary skill in the art 
at the time of the invention to have not used a microlens on non-standard photosite eO depicted in 
Fig. 1 as suggested by Park and to have used different microlenses for photosites el-e3 as 
suggested by Mayar in order to reduce the number of microlenses needed to be formed on the 
image sensor. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian C. Genco who can be reached by phone at 703-305-7881 or 
by fax at 703-746-8325. The examiner can normally be reached on Monday thru Thursday 
7:30am to 4:30 pm and every other Friday 7:30am to 3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Christensen can be reached on 703-308-9644. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 
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Any inquiry of a general nature or relating to the status of this application or proceeding 

should be directed to the customer service office whose telephone number is 703-308-4357. 

Brian C Genco 
Examiner 
Art Unit 2615 

January 26, 2004 
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